Maxi-Min discriminant analysis via online learning
نویسندگان
چکیده
Linear Discriminant Analysis (LDA) is an important dimensionality reduction algorithm, but its performance is usually limited on multi-class data. Such limitation is incurred by the fact that LDA actually maximizes the average divergence among classes, whereby similar classes with smaller divergence tend to be merged in the subspace. To address this problem, we propose a novel dimensionality reduction method called Maxi-Min Discriminant Analysis (MMDA). In contrast to the traditional LDA, MMDA attempts to find a low-dimensional subspace by maximizing the minimal (worst-case) divergence among classes. This "minimal" setting overcomes the problem of LDA that tends to merge similar classes with smaller divergence when used for multi-class data. We formulate MMDA as a convex problem and further as a large-margin learning problem. One key contribution is that we design an efficient online learning algorithm to solve the involved problem, making the proposed method applicable to large scale data. Experimental results on various datasets demonstrate the efficiency and the efficacy of our proposed method against five other competitive approaches, and the scalability to the data with thousands of classes.
منابع مشابه
Maxi-Min Margin Machine: Learning Large Margin Classifiers Locally and Globally
In this paper, we propose a novel large margin classifier, called the maxi-min margin machine M(4). This model learns the decision boundary both locally and globally. In comparison, other large margin classifiers construct separating hyperplanes only either locally or globally. For example, a state-of-the-art large margin classifier, the support vector machine (SVM), considers data only locally...
متن کاملOnline Monitoring and Fault Diagnosis of Multivariate-attribute Process Mean Using Neural Networks and Discriminant Analysis Technique
In some statistical process control applications, the process data are not Normally distributed and characterized by the combination of both variable and attributes quality characteristics. Despite different methods which are proposed separately for monitoring multivariate and multi-attribute processes, only few methods are available in the literature for monitoring multivariate-attribute proce...
متن کاملAdaptive Learning Rate for Online Linear Discriminant Classifiers
We propose a strategy for updating the learning rate parameter of online linear classifiers for streaming data with concept drift. The change in the learning rate is guided by the change in a running estimate of the classification error. In addition, we propose an online version of the standard linear discriminant classifier (O-LDC) in which the inverse of the common covariance matrix is update...
متن کاملGeneralized locality preserving Maxi-Min Margin Machine
Research on large margin classifiers from the "local" and "global" view has become an active topic in machine learning and pattern recognition. Inspired from the typical local and global learning machine Maxi-Min Margin Machine (M⁴) and the idea of the Locality Preserving Projections (LPP), we propose a novel large margin classifier, the Generalized Locality Preserving Maxi-Min Margin Machine (...
متن کاملLinkages between motivation, self-efficacy, self-regulated learning and preferences for traditional learning environments or those with an online component
This study assessed 96 law school students’ preferences for online, hybrid, or traditional learning environments, and their reasons for these preferences, learning strategies, and motivational orientations. A discriminant analysis revealed that non-traditional learning environment familiarity, self-efficacy, and employment status were the strongest predictors of preferences for non-traditional ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Neural networks : the official journal of the International Neural Network Society
دوره 34 شماره
صفحات -
تاریخ انتشار 2012